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ABSTRACT 

This issues paper, seventh in a series of eight, is intended 
to distill formative evaluation questions on topics that are central to the 
development of the higher and further education information environment in 
the United Kingdom. The aim of this issues paper is to provide a description 
of the approach of the Formative Evaluation of the Distributed National 
Electronic Resource (EDNER) Project to understanding implicit theories of 
change to allow project teams (learning resource development teams) to carry 
out their own exercises in "surfacing" the assumptions embedded in their own 
work. Revealing such assumptions results in airing differences, improving 
consensus, and enhancing the internal logic of the project. Surfacing the 
theory of change in a project begins with each team member writing down a 
vision of the "outcome of interest, " and the circulation of these documents 
to all team members. A logic map is then produced of the project's theory of 
change, and the directed links between inputs and goals and outputs are 
explains. The work of explaining these links is what brings the project *s 
implicit theory of change to the surface. The map is revised as necessary to 
keep the reflection of the project current. (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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The aim of this Issues Paper is to provide enough of a description of our 
approach to understanding implicit theories of change to allow project 
teams to carry out their own exercise in 'surfacing' the assumptions 
embedded in their own work. The point of carrying out such an exercise is to 
reveal such assumptions, so that differences within and around the project 
team can be aired, consensus improved and the internal logic of the project 
enhanced. The approach described here builds on the work of John Nash 
(Stanford Learning Lab), Leo Plugge and Anneke Eurelings (until recently at 
the University of Maastricht), and Helge Stromdahl (Royal Institute of 
Technology, Stockholm) who are among the first people we know to have 
tried out 'theory-based evaluation' or 'theory-anchored evaluation' in the 
field of learning technologies (see Nash et al, 2000; Stromdahl & Langerth- 
Zettermon, 2001). 

Surfacing a project's theory of change involves the following steps: 

1 . Each member of the project team (and ideally each important 
stakeholder) should write down their vision of the project's 'outcome of 
interest'. One way to do this is to get each person to write a document in 
response to a 'History of the Future' Exercise. (See Box 1 .) 

2. Copies of the documents thus produced should be given to all members 
of the project team (and ideally each important stakeholder) and a 
meeting held to identify common elements, identify key differences and 
work towards consensus about a definition of the main outcome or 
outcomes of interest. This should be written down. 

3. At the same or a later meeting, project team members and stakeholders - 
usually with the aid of a facilitator - should try to create a logic map of the 
project's theory of change. (For more about how to use logic models to 
help a project tell its story about change see McLaughlin & Jordan, 1 998.) 
An example of a project logic map is given in Figure 1 

4. The logic map in Figure 1 consists of project outputs (ellipses, given on the 
right of the map), project activities or inputs (rectangles, given on the left 
of the map) and intermediate goals (rounded rectangles, in the centre of 
the map). The definition of the project outcomes begins (and often ends) 
with agreement about the main outcome(s) of interest. The inputs are the 
'big ideas' brought together at the outset of the project. They may reflect 
resources and activities in the real world or theoretical constructs. The 
intermediate goals represent states of affairs that bridge between inputs 
and outputs. Their creation may well be the principal work of the project. 
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5. Once an agreed project logic map has been produced, explain the 
directed links between the inputs and the intermediate goals and 
between the intermediate goals and the outputs. (It may help to number 
these links on the map and write paragraphs about each link in an 
accompanying document.) The work involved in explaining these links is 
what 'brings to the surface ' the project's implicit theory of change. 

6. Over a period, revisit the map, adding detail as appropriate and 
amending elements in light of experience, changing circumstances etc. 
Be sure to retain a copy of each main version of the map and its 
accompanying documentation. 



Box 1 Eliciting ‘outcomes of interest’ 

To facilitate this process for complex projects, we propose that the project staff 
write a history of the future. 

Imagine that your Intervention project is completed and that it succeeded In all of 
its goals. You are to appear tomorrow at a press conference to explain what you 
have accomplished. Write a press release for distributing at this meeting, 
explaining in a few paragraphs what it is that you have accomplished, who is 
benefiting from this, why It’s important (that is, what problem it solves and why 
this problem needed to be solved), and what it was that you did that led to or 
caused this success. 

After Vanezky (2000) - see Nash et al (2000). 



Inputs Intermediate Goals 



Outputs 




Fig 1 Example project logic map (after Nash et al, 2000) 
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EDNER Key Issues papers are intended to distil formative evaluation questions 
on topics which are central to the development of the UK's higher and further 
education Information Environment. They are presented as short check-lists of 
key questions and are addressed to developers and practitioners. Feedback 

to the EDNER team is welcomed. 



Please address enquiries and comments to the EDNER Project Team at 

cerlim@mmu.ac.uk 

EDNER is being undertaken by CERLiM at the Manchester Metropoiitan University with CSALT at Lancaster 

University 
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